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Abstract — The serial concatenation of a repetition code with 
two or more accumulators has the advantage of a simple encoder 
structure. Furthermore, the resulting ensemble is asymptotically 
good and exhibits minimum distance growing linearly with block 
length. However, in practice these codes cannot be decoded by 
a maximum likelihood decoder, and iterative decoding schemes 
must be employed. For low-density parity-check codes, the notion 
of trapping sets has been introduced to estimate the performance 
of these codes under iterative message passing decoding. In this 
paper, we present a closed form finite length ensemble trapping 
set enumerator for repeat multiple accumulate codes by creating 
a trellis representation of trapping sets. We also obtain the 
asymptotic expressions when the block length tends to infinity 
and evaluate them numerically. 

L Introduction 

Turbo-like codes flj, as well as LDPC codes El, can 
perform close to the Shannon limit using suboptimal iterative 
decoding schemes. However, these codes typically exhibit an 
error floor at medium to high signal-to-noise ratios (SNRs). 
In O, the height of the error floor of LDPC codes was linked 
to so-called "near codewords". Later, in IH, this concept was 
generalized to trapping sets, substructures in the Tanner graph 
of a code that may cause the iterative message passing decoder 
to fail. For certain LDPC codes, small trapping sets, rather 
than the minimum distance of the code, dominate the error 
floor performance. 

Asymptotic spectra of trapping sets in LDPC code ensem- 
bles were computed in (51 for regular and irregular LDPC 
codes and in 01 for protograph-based codes. It was shown 
that there exist LDPC codes that exhibit a minimum trapping 
set size growing linearly with block length, for certain types 
of trapping sets. 

In turbo-like codes, the concatenation of simple component 
codes through interleavers can lead to powerful code construc- 
tions. The simplest examples are repeat multiple accumulate 
(RMA) codes. These codes have a low encoding complexity 
of 0(1) and can be decoded using relatively few iterations. 

This work was partly supported by NSF grants CCF05- 15012, 
CCF08-30666, NASA grant NNX07AK536, and the Marie Curie Intra- 
European Fellowship within the 6th European Community Framework Pro- 
gramme. 



Furthermore, it has been shown in 171 and IH that the double 
serially concatenated repeat accumulate accumulate (RAA) 
code of rate 1/3 or smaller is asymptotically good and exhibits 
minimum distance growing linearly with block length. 

Like LDPC codes, turbo-like codes are decoded in an itera- 
tive fashion. Commonly, the component codes are decoded 
with a maximum a posteriori probability (MAP) decoding 
algorithm and the extrinsic information provided by a com- 
ponent decoder functions as a priori information for another. 
For RMA codes, the turbo decoder can be represented as a 
message passing decoder 13, similar to the belief propagation 
decoder, albeit with a different message passing schedule. 
Thus the turbo decoder may also be susceptible to trapping 
sets. To predict the error floor of a code one generally needs 
to have full knowledge of the trapping sets that dominate the 
error floor, i.e., one needs to know their graph structure and 
enumerate their multiplicities, and to find the probability that 
the decoder gets trapped in a particular set. The latter not only 
depends on the graph structure of the trapping set but also on 
the channel model, the decoding algorithm, and the particular 
decoder implementation that is used. 

In this paper we address the first part of the problem, the 
enumeration of subgraphs in an RAA code. We derive a closed 
form trapping set enumerator (TSE) for general (a, b) trapping 
sets, as defined in [4J and [5|. A general (a, b) trapping set for 
a given Tanner graph is a set of a variable nodes that induces 
a subgraph containing b odd degree check nodes, which can 
be thought of as unsatisfied checks, and an arbitrary number 
of even degree check nodes. If there are only a few unsatisfied 
check nodes and a sufficiently large number of erroneous 
variable nodes, the iterative message passing decoder may not 
be able to correct the erroneous nodes. The TSE is the average 
number of (a, b) trapping sets in the ensemble composed of 
all possible interleaver realizations. We also derive asymptotic 
expressions for the TSE and analyze them. 

II. Trapping set enumerators for repeat 

ACCUMULATE ACCUMULATE CODE ENSEMBLES 

The encoder structure of an RAA code C^^^ is shown 
in Fig. [T] It is a serial concatenation of a repetition code 
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Fig. 2. Block diagram and factor graph representation of an accumulate 
code. 
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To proceed, we must define the trapping set enumerators 
of the component codes ^^fjw a° 6f ^ ~ ^J^' 

Since there are no check nodes in layer 1 of the factor graph, 
= (^) input-output weight enumerator (lOWE) 
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Fig. 1. Block diagram and factor graph {q = 3) of an RAA encoder. The 
circles represent variable nodes and the boxes check nodes, respectively. 

Qrep ^^^Q j^^^^ = l/q and two identical rate-1, memory- 1, 
accumulate codes Cf^^, / = {1,2}, with generator polynomials 
g{D) = 1/(1 + 1)), connected by interleavers tti and 7T2. For 
the repetition code C^^p, we denote the binary input sequence 
of length i^T by u = [ui, . . . ^uk] and the binary output 
sequence of length N by x^^p = [x^®^, . . . , xj^^]. Likewise, for 
encoder Cf^^, = [v[^ . . . ^ v^j^] and = [x^, . . . , x^] denote 
the input sequence and the codeword, respectively, where both 
are of length N. Note that = 7ri(x^^P) and = 7r2(x^). 
The overall code rate is R = K/N. 

The factor graph of an RAA code is also depicted in Fig. [T] 
for a repetition factor of g = 3. The circles represent variable 
nodes while the boxes represent check nodes. The information 
symbols u correspond to the variable nodes of the first layer 
and their degree is equal to the repetition factor q. The variable 
nodes of the second layer correspond to the output of the first 
accumulator x^, and the variable nodes of the third layer to 
the output of the second accumulator x^, respectively. The 
input bits of the accumulators are represented by the variable 
nodes of the next higher layer. Only the variable nodes in the 
third layer are transmitted through the channel. This implies 
that, initially, only variable nodes in the third layer can be in 
error, while the others have a neutral initial value. However, 
in the first decoding iteration, the variable nodes in layers 1 
and 2 get values assigned based on the received sequence. 
If there are trapping sets containing variable nodes in those 
layers, erroneous values that were assigned during the first 
iteration may never be corrected and may cause the iterative 
decoder to fail. Therefore, we consider the whole graph when 
enumeratin^^for trapping sets. 

Let ^ be the ensemble-average TSE of an RAA code 
ensemble, i.e., the average number of (a, b) trapping sets. With 
reference to Fig.[T] we denote by w the number of information 
bits that participate in an (a, b) trapping set of C^^^. Also, let 
a], a^, and bi be the number of variable nodes corresponding 
to input bits, the number of variable nodes corresponding to 
code bits, and the number of unsatisfied checks, respectively, 
of code Cf^^ involved in an (a, b) trapping set of C^^^. 



^rep input weight w and output weight qw, while A ^ ^ 
is the input-output trapping set enumerator (lOTSE) of code 
Qacc^ denoting the number of trapping sets in Cf^^ consisting 
of al input variable nodes (i.e., variable nodes corresponding 
to information bits), output variable nodes (i.e., variable 
nodes corresponding to code bits), and bi unsatisfied checks. 

With these definitions, the ensemble average TSE ^ 
can be computed using the uniform interleaver concept iTOl 
as: 
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, is called the ensemble-average condi- 
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where A^„ ^01, 
tional TSE. 

The evaluation of ([T]) requires the computation of A \ „ , 
which will be presented in the next section. The extension 
of ([B to more than two serially concatenated accumulators is 
straightforward. 

in. Input-output trapping set enumerator for the 

ACCUMULATE CODE 
In the following, we address the computation of the lOTSE 

^acc 

A \ ^ of an accumulate code by considering an equivalent 
trellis representation of trapping sets in the factor graph. In 
Fig. [21 the block diagram of an accumulate code and a single 
section of the corresponding factor graph are depicted. From 
the figure we obtain the following relation: 



Vk = Xk-l ^Xk. 



(2) 



Four different 3-tuples {vk-,Xk-i-,Xk) are possible, namely 
(0,0,0), (0,1,1), (1,0,1), and (1,1,0), such that the parity 
check is satisfied. Their factor graph representations are shown 
in Fig. [3a), where black circles represent non-zero symbols 
and empty circles represent zero symbols. Now consider an 
(a, 6) trapping set of C^^^, and assume that (some) of the 
variable nodes of accumulate code C^^^ corresponding to 
(vk^Xk-i^Xk) participate in the trapping set and cause an 
unsatisfied check. Again, only four different configurations are 
possible. They are depicted in Fig. [3b), where black circles 
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Fig. 3. Factor graph representations of an accumulate code. 



where m and n must satisfy the constraints 
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correspond to erroneous symbols and a black box means that 
the check is unsatisfied. Note that all possible trapping sets 
can be obtained by properly combining the eight factor graph 
sections of Fig. [3] 

For enumeration purposes, it is simpler to refer to an 
equivalent trellis representation. Assign to the variable nodes 
and the check nodes in Fig. [3] that participate in a trapping 
set (the black circles and boxes) the value 1. Then the eight 
factor graph sections in Fig. [3] can be conveniently represented 
by the equivalent trellis section of Fig. IH We call this the 
extended trellis section since it extends the standard trellis 
section of an accumulate code to include all possible trapping 
sets. Each edge between two trellis states is labeled with a 
binary 3-tuple Si/c/so, where Si denotes the input symbol, 
So denotes the output symbol, and c is 1 if the check node 
in the corresponding equivalent factor graph representation 
is unsatisfied. The four labels in black correspond to the 
four configurations of Fig. Oa) and define the standard trellis 
section of an accumulate code, while the four labels in 
red correspond to the four configurations of Fig. Hb). Now 
the lOTSE of the accumulate code can be computed from 
the trellis representation of Fig. |4l by considering a trellis 
consisting of N concatenated trellis sections like the one in 
Fig. Hand enumerating all possible paths. The lOTSE is given 
in closed form in the following Theorem. 

Theorem 1. Let {a\ a^, b) be a trapping set with a* informa- 
tion variable nodes, code variable nodes, and b unsatisfied 
checks. The input-output trapping set enumerator (lOTSE) 
for the rate-1, memory-1, convolutional encoder C^^^ with 
generator polynomial g{D) = 1/(1 + D), terminated to the 
all-zero state at the end of the trellis, and with input and output 
block length N, can be given in closed form as: 
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Proof: Consider the extended trellis section of the encoder 
g{D) = -\- D) in Fig. |4l Denote by n the number of 
length-one error events 1/1/0 from the zero state to the zero 
state, called type-1 error events, and by m the number of 
error events that leave the zero state, and remerge later to the 
zero state, called type-2 error events. Further, let a\ aP , and 
b be the number of information variable nodes, the number of 
code variable nodes, and the number of unsatisfied checks, 
respectively, participating in the trapping set. Also, let 
denote the total input weight associated with the transitions 
— > 1 (from state zero to state one) and 1^0 (from state 
one to state zero) in the m type-2 error events. 

Only type-2 error events are responsible for the weight at 
the output of the accumulator. From lITTIl . we know that the 
number of permutations of m type-2 error events resulting in 
an output weight of is 

- /a^ - 1^ 



m 
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m - 



(Here, the transitions away from and back to the zero state are 
not only caused by the input weight but also by 2m — 
unsatisfied check nodes.) The m type-2 error events include 
— m transitions from the one state to the one state (1 1), 
and the input weight associated with the transitions 1 ^ 1 is 

-wK (5) 



a 



Moreover, the following equality holds: 

a' -b 



m. 



(6) 



Also, due to termination, a* + 6 is even. From ^ and © it 
now follows that wi^i = —n — m. This weight can 
be ordered in ( ai+b~^ ) different ways, which gives the 

^ — f n—m^ ^ 

third binomial coefficient in (O. On the other hand, there are 
N — — m transitions from the zero state to the zero state 
with an associated input weight n. Therefore, we obtain the 
term (^~'^ ~^)- The last binomial coefficient in © results 
from the ordering of the ones in the 2m transitions 0^1 
and 1 ^ 0, in ( gi-T ^ ^) ways. 

To summarize, the number of paths in the extended trellis 
consisting of n type-1 error events and m type-2 error events 
is given by: 
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The result for the encoder g{D) = 1/(1 + 1^) follows by 
summing over all possible values of n and m. ■ 

Corollary 1. For b = 0, the expression in (0) reduces to the 
well-known lOWEfor the rate-1, memory-1, accumulate code 



IV. Asymptotic ensemble trapping set enumerator 

In order to determine the asymptotic spectral shape of the 
trapping sets associated with a particular code ensemble, as 
the block length N tends to infinity, we define the normalized 
logarithmic asymptotic TSE r^(a, (3) of a code ensemble C as 



r (a, ^) = limsup 



(7) 



where a = a/N, f3 = b/N, and the supremum is taken over all 
intermediate variables. We also define the functions /^"^""^ and 
as the asymptotic behavior of the lOWE of a repeat code 
and the asymptotic behavior of the lOTSE of an accumulate 
code Cf^^, respectively: 

In 4<^"'' 
f (ij) = lim — ^ 



N^oo iV 

where u; = w/K,ai= a\/N, = af/AT, and A = ^//^'. 

Using Stirling's approximation for binomial coefficients 
(fe) ""-^ e^^(-), where EI(-) is the binary entropy function 
with natural logarithms, the functions in ([8]) can be written as: 



(9) 



and 
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(10) 

where we have defined the normalized quantities jii = rui/N 
and vi = ni/Ni. 

Then, using (ISlfTOl) and ([B in (|7]), the asymptotic TSE of a 
code ensemble C^^^ can be written as: 



= sup 

/3=/3i+/32 



(11) 



with the constraints a = ^ + + and /3 = /^i + /?2- 

V. Numerical Evaluation 

In this section, we present a numerical evaluation of (fTTI) . 
Following 1 6 1, in the curves for the asymptotic TSE that we 
present, we keep the ratio A = of unsatisfied check nodes 
to erroneous variable nodes constant and compute r(a, Aa) 
for varying values of a. In Fig. [5j the unsatisfied checks in 
the RAA code ensemble are equally distributed between the 
middle and inner accumulator, i.e., Pi = = P/2. For 
A = 0, when no unsatisfied checks are present in the factor 
graph, the spectral shape r(a, 0) exhibits a zero stretch in the 
beginning and turns positive when the number of codewords 
with normalized weight a starts to grow exponentially in 
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Fig. 5. Asymptotic TSE for different values of A and Pi = [32 = /^/2. 
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Fig. 6. Asymptotic TSE for different fractions f3i/f3. 

N with increasing a. The presence of unsatisfied checks in 
the factor graph results in a positive initial slope, and we 
observe a quasi-linear increasing first section of the curve, until 
there is a discontinuity in the slope. In the second section, 
the slope of the curve is similar for all values of A, and 
the curve shifts to the left with increasing A. Also, as the 
fraction of unsatisfied checks A increases, the slope in the 
first section also increases. Because of the large number 
of parameters involved in taking the supremum in (fTTI) . it 
is difficult to draw general conclusions about the trapping 
set structures that are most likely to cause decoding failures. 
The structure of a trapping set is greatly influenced by the 
choice of these parameters. We are primarily concerned with 
trapping set configurations that lead to decoding errors and 
this requires co > 0. The choice of the parameters (3i and ^2 
determines how many unsatisfied checks are associated with 
the middle and inner accumulator, respectively. For instance, 
in the extreme case of Pi = P and = 0, all the unsatisfied 
checks are associated with the middle accumulator, and there 
are no unsatisfied checks in the graph of the inner accumulator. 

In Fig. [6] we vary the ratio Pi/P, the fraction of unsatisfied 
check nodes associated with the middle accumulator in the 
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Fig. 7. Asymptotic TSE of the RAA code ensemble for different repetition 
factors q. 
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Fig. 8. Asymptotic TSE for the RAA, RAA A, and RAAAA code ensembles. 

RAA code ensemble. For larger (3i/(3, when relatively more 
unsatisfied check nodes are present in the middle accumulator, 
the slope in the first section is smaller and the influence of 
varying Pi/ [5 on the slope becomes greater as /^^ gets closer 
to one. However, the influence that varying A has on the slope 
is much greater than the influence of varying f3i / (3. Also, for 
larger values of A, the variance of the curves with Pi/P is 
greater. 

In Figs. [7] and [S] we display the influence of the repeti- 
tion factor q and the number of concatenated accumulators, 
respectively, on the shape of the asymptotic TSE. In both 
cases the greatest effect on the slope in the first section 
was observed when pi = i.e., when all unsatisfied check 
nodes are associated with the outermost accumulator. In the 
other extreme case, when all the unsatisfied check nodes are 
associated with the inner accumulator, the slope in the first 
section did not change. In Fig. [71 the reduction in slope in 
the first section caused by increasing the repetition factor q 
is only marginal and the curves almost lie on top of each 
other. However, increasing the number of serially concatenated 
accumulators decreases the slope in the first section, as can be 
seen in Fig. [H Finally, we note that increasing the repetition 
factor q or adding more accumulators increases the minimum 
distance of the code, and thus the transition from the quasi- 
linear section of the asymptotic TSE to the more steeply 
increasing section takes place at higher values of a. 



VI. Conclusions 

We have presented a simple closed form method to enu- 
merate general (a, b) trapping sets for RAA code ensembles. 
The trapping set enumerator is first obtained for finite block 
lengths TV and its asymptotic expression is derived by letting 
N go to infinity. Similar to and 0, we observe that, 
when unsatisfied check nodes are present in the factor graph, 
the asymptotic TSE lies strictly above the asymptotic spectral 
shape for the case when no unsatisfied check nodes exist in 
the graph. Although the RAA code ensemble is asymptotically 
good and exhibits minimum distance growing linearly with 
block length, in contrast to regular and some protograph- 
based irregular LDPC codes, there exists no region where 
the minimum trapping set size grows linearly with block 
length. It can, at best, grow only sublinearly in the block 
length, since the asymptotic TSE of the RAA code ensemble 
is always positive if unsatisfied check nodes are present in 
the graph. While the method presented in this paper allows 
us to enumerate all general (a, b) trapping sets, the influence 
that these trapping sets have on the error floor must still 
be evaluated separately. As noted earlier, the probability that 
the decoder gets stuck in particular types of trapping sets 
depends on the channel, the decoding algorithm, and the 
particular decoder implementation. In future work we hope to 
evaluate this probability for the turbo decoder and the belief 
propagation decoder, in order to obtain a reliable estimate of 
the height of the error floor for RMA codes. 
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